1. Field of the Invention
The present invention relates to the field of computer graphics and, in particular, to a method and apparatus for tone mapping for motion pictures.
2. Description of the Related Art
Displaying images having wide variations in illumination while preserving visually important features has proven to be challenging. This problem exists for both rendered images and real images captured on film or with a digital sensor in a digital camera. For example, illuminating an image of a person standing on a beach with the setting sun in the background, while preserving the beauty of the sunset, is a particularly difficult problem to solve. The challenge is to capture detail in the dark foreground as well as in the bright background. For still images, photographers go to great lengths to find ways to illuminate the foreground subject in a manner that looks natural, while preserving the background color and illumination. For motion pictures, a director of photography can carefully apply lighting effects to a shot to achieve the desired effect. However, additional problems arise when attempting to automate these processes.
The above issues are exacerbated when viewing high-dynamic range (HDR) digital images, having wide variations in illumination, on a limited-dynamic range (LDR) display. The human eye can perceive a large dynamic range of brightnesses, and an HDR image attempts to duplicate this dynamic range. Information stored in HDR images often corresponds to the physical illumination values, such as luminance, radiance, contrast, saturation, and the like. HDR images are often called “scene-referred” images. By synthesizing multiple images having overlapping illumination ranges in brightness, user are able to generate HDR images with essentially unlimited dynamic range. By contrast, display screen images, prints, and other output media have a limited dynamic range. Such displayed images are often referred to “device-referred” or “output-referred” images. Because of the disparity in the dynamic ranges of HDR images and LDR displays, various attempts have been made to “trick” the eye into perceiving greater dynamic ranges on LDR displays.
One approach for converting HDR images into a form more viewable on LDR displays is generally referred to as “tone mapping.” One conventional tone mapping technique allows a user to select a subset of the entire illumination value range to map to an LDR display. This technique allows the user to “bring up” the detail in the lower end of the illumination range and “pull down” the detail in the higher end of the illumination range to fit within that pre-defined subset. Thus, the range of illumination values of an LDR image can be shifted up or down in an attempt to capture at least some of the detail in both high and low levels of illumination in an HDR image. However, one drawback to tone mapping is that the selected range often includes an area that is too saturated or an area that is too dark, causing the resulting image to look flat and dull. This problem is often referred to as “dynamic range compression.”
Some more recent tone mapping techniques have tried to overcome the limitations of dynamic range compression by relying on global operators that implement HDR radiance maps to translate the HDR image to the LDR display using the same tone mapping parameters for each pixel of the output image. Once an optimal mapping has been determined for the entire image, each pixel is mapped using the same mapping function. Some user input is then required to fine-tune certain areas of the image to correct any resulting visual artifacts (e.g., a portion of the image may be too dark to be visible and so the illumination values in this portion may need to be increased). Since each image must be mapped and then locally adjusted, “global” tone mapping is usually an arduous and time-consuming task.
To address the problems with global tone mapping, other tone mapping operators, referred to a “local” tone mapping operators, have been developed where local gradients are preserved. With local tone mapping operators, illumination value gain changes and brightness levels of surrounding pixels influence the mapping for a particular pixel, thereby addressing the artifacts introduced using global tone map operators. However, a problem often encountered with using local tone mapping operators is the generation of a “halo” or “ringing” effect. For example, a brightly-colored object may be set against a dark background. When tone mapping such an image using a local tone mapping operator, the illumination values of the dark pixels immediately surrounding the object are influenced by the brightness of the object, creating an artifact that resembles a halo or glow that surrounds the object. These halo effects can be reduced manually by a user after the local tone mapping operation has completed. Again, manual adjustment can become cumbersome and time-consuming.
Each of these problems set forth above is exacerbated in the context of motion pictures, because variations between sequential images (e.g., frames) change over time. For example, processing individual images in a sequence using prior art tone mapping techniques would generally result in variations in tone mapping from image to image, causing the images to “pop” or, alternatively, causing halo effects that move through the frames with an object.
As the foregoing illustrates, there is a need in the art for an improved tone mapping technique that is also suitable for motion pictures.